WORLD INTELLECTUAL PROPERTY ORGANIZATION 
Internationa] Bureau 



INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
G09B 7/00 


Al 


(11) International Publication Number: WO 99/24953 
(43) International Publication Date: 20 May 1999 (20.05.99) 


(21) International Application Number: PCT/US98/23979 

(22) International FUing Date: 10 November 1998 (10.1 1.98) 

(30) Priority Data: 

08/968,681 12 November 1997 (12.1 1.97) US 

(71)(72) Applicants and Inventors: HO, Chi, Fai [US/US]; 4816 
Cabello Court, Union City, CA 94587 (US). TONG, Peter, 
P. [US/US]; 1807 Limetree Lane, Mountain View, CA 
94040 (US). 

(74) Common Representative: TONG, Peter, P.; 1807 Limetree 
Lane, Mountain View, CA 94040 (US). 


(81) Designated States: AL, AM, AT, AU, AZ, BB, BG, BR, BY, 
CA, CH, CN, CZ, DE, DK, EE, ES, FI, GB, GE, HU, IL, 
IS, JP, KE, KG, KP, KR, KZ, LK, LR, LS, LT, LU, LV, 
MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, 
SD, SE, SG, SI, SK, TJ, TM, TR, TT, UA, UG, UZ, VN, 
ARIPO patent (GH, GM, KE, LS, MW, SD, SZ, UG, ZW), 
Eurasian patent (AM, AZ, BY, KG. KZ, MD, RU, TJ, TM), 
European patent (AT, BE, CH, CY, DE, DK, ES, FI, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE), OAPI patent (BF, 
BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, 
TD, TG). 

Published 

With international search report 



(54) Title: A LEARNING SYSTEM AND METHOD BASED ON REVIEW 



(57) Abstract 

A system and method to provide dy- 
namic reviews for individual students. The 
system (100) selects an un-learnt line-item 
(200), and generates detailed learning mate- 
rials for the un-leamt line-item (204); then 
selects a learnt line-item based on one or 
more learnt-line-item-selection rules (206), 
and generates summarized-leaming materials 
on the selected learnt line-item for the student 
to review (208). After presenting the gener- 
ated learning materials to the student for an 
un-leamt line-item, the system assesses the 
students understanding of the line-item. If 
the assessment is satisfactory, the un-leamt 
line-item is re-classified as a learnt line-item. 
The system also accesses a line-item prob- 
ability value (354), which is used to select 
a line-item. If the line-item is a learnt 
line-item, which is selected based on one or 
more leamt-line-selection rules, the invention 
generates summarized learning materials for 
the student (360). The invention then repeats 
and selects another line-item (356). 
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WO 99/24953 PCT/US98/23979 
A LEARNING SYSTEM AND METHOD BASED ON REVIEW 



BACKGROUND OF THE INVENTION 

5 The present invention relates generally to education, and more particularly to 

using reviews to enrich the learning process in a computer-aided educational system and 
method. 

Review is very important in any educational or training systems. Most of us have 
encountered situations where we have forgotten what we have learnt, and would 

10 appreciate some forms of review to refresh our memory. As subject matter becomes more 
complicated, our need for review increases, especially when we are learning an inter- 
disciplinary subject, or solving inter-disciplinary problems. For example, when we 
encounter the problem of performance degradation due to temperature increase in an 
integrated circuit, probably some of us would like to design a heat-sink with fins on the 

15 integrated circuit, and would like to have a review on the subject of heat-flow on a heat- 
sink with different types of fins. 

In a typical computer-aided educational system, there is no review. Those systems 
assume the students remember everything they have learnt, and just keep on teaching new 
areas. If a student wants to review certain subject matter, she can always go back and 

20 search for it. However, most students generally do not go back; they typically proceed 
forward and believe that once they have learnt a subject, they will never forget. As 
forgetfulness and lack in understanding start to build up, not for long a typical student 
will be totally lost, with his interest in learning the subject significantly diminished. 

There is one exception to the typical educational system; it provides students with 

25 reviews. However, those reviews are not dynamic - they are just fixed reviews, which 
are not personalized. Such a system provides a student a set of study-plan. After the 
system has presented the study-plan to the student, the system automatically provides a 
summary of what the student has just learnt. Such an approach is not useful. Since the 
student has just studied the material, the probability of the student to forget is quite low. 

30 Also, the student might understand fully what she has just studied. The system does not 
know, and does not care. The reviews are fixed. Under no condition would the system 
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not present the review to the student. Such fixed reviews not catered to the needs of 
individual students are not very helpful to the students. 

It should be obvious from the foregoing that there is still a need for a computer- 
aided educational system and method that is catered specifically to the needs of individual 
5 students, or the characteristics of the subject, through dynamic review so as to reinforce 
each student's learning in a subject. 
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SUMMARY OF THE INVENTION 



The invented computer-aided educational system and method provide dynamic 
reviews for individual students. After a student has learnt certain areas in a subject, 
5 summarized learning materials on those areas are selectively generated for the student so 
as to reinforce the students learning in the subject through reviews. Unlike prior systems, 
the reviews in the present invention are dynamic; they are specifically tailored to the 
needs of individual students, or the characteristics of the subject. 

The present invention is applicable to any subject that can be taught by a 
10 computer. A subject in the present invention is divided into areas that the student has 

learnt and those that the student has not learnt; they are referred to as learnt line-items and 
un-learnt line-items respectively. The line-items are stored in a line-item storage- 
medium. 

In one embodiment, the present invention selects an un-learnt line-item, and 

15 generates detailed learning materials for the un-learnt line-item. Then a learnt line-item is 
selected based on one or more learnt-line-item-selection rules, depending on factors such 
as the time elapsed from the time when the student learnt that line-item, the level 
achieved by the student in learning that line-item, its difficulty level, whether that learnt 
line-item is related to the selected un-learnt line-item, and whether that learnt line-item 

20 has been selected before. Then, the invention generates summarized-leaming materials 
on the selected learnt line-item for the student to review. 

Different materials are applicable for learning. The summarized-leaming 
materials include study-plans. The detailed learning materials may further include 
questions, in addition to study-plans. 

25 For an un-learnt line-item, after presenting the generated learning material to the 

student, the present invention assesses the student's understanding in the line-item. If the 
assessment is satisfactory, the un-learnt line-item is re-classified as a learnt line-item. 

In another embodiment, the present invention accesses a line-item probability 
value, which is used to select a line-item. If the line-item is an un-learnt line-item, the 

30 invention generates detailed learning materials for the student; and if the student is 
successful in learning the materials, the invention re-classifies the line-item to a learnt 
line-item. If the line-item is a learnt line-item, the invention generates summarized 



I » 
I I 

WO 99/24953 PCT/US98/23979 

learning materials for the student. The invention then repeats and selects another line- 
item. 

Other aspects and advantages of the present invention will become apparent from 
the following detailed description, which, when taken in conjunction with the 
5 accompanying drawings, illustrates by way of example the principles of the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG..1 shows one embodiment of a line-item entry in the present invention. 
FIG. 2 shows an embodiment of the present invention. 
FIGS. 3A-B show one embodiment of a system implementing the present 
5 invention. 

FIG. 4 shows a set of steps to implement one embodiment of the present invention 
FIG. 5 shows an example of a difficulty hierarchy in the present invention. 
FIG. 6 shows a set of steps to generate a detailed learning material in the present 
invention. 

10 FIG. 7 shows a number of criteria to select a learnt line-item in the present 

invention. 

FIG. 8 shows a set of steps to implement another embodiment of the present 
invention. 

FIG. 9 shows a set of steps to select a line-item based on a line-item probability. 
1 5 Same numerals in FIGS. 1 -9 are assigned to similar elements in all the figures. 

Embodiments of the invention are discussed below with reference to FIGS. 1-9. 
However, those skilled in the art will readily appreciate that the detailed description given 
herein with respect to these figures is for explanatory purposes as the invention extends 
beyond these limited embodiments. 
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DETAILED DESCRIPTION OF THE INVENTION 



The present invention is applicable to teach any subject that can be taught by a 
computer. The subject may be a subject taught in one-semester or in a year, or in just one 
5 class session. The subject may cover inter-disciplinary areas, such as electrical 

engineering and thermodynamics, or computer networking and programming techniques. 
In the following, mathematics is the subject used to illustrate the present invention. 

In one embodiment, the subject is divided into major- topics, with each major- 
topic subdivided into minor-topics, and with each minor-topic further subdivided into 
10 line-items. Each line-item typically covers one well-defined area in the subject. In 

another embodiment, the subject is further divided into more levels below the line-items; 
and in a third embodiment, the subject is just divided into line-items. 

As an example of line-items, if the major-topic is high school algebra, then it can 
be divided into the following line-items, with bracketed terms served as comments: 



15 



20 



High School Algebra (the major-topic) 
(Minor-topics under the major-topic) 
Decimal Numbers 
Polynomials 
Linear Equations 
Quadratic Equations 



25 



30 



Integers 

(Line-items under the minor-topic of integers) 
Addition & Subtraction (Difficulty level 1) 
Multiplication (Difficulty level 2) 
Division (Difficulty level 2) 
Prime Numbers (Difficulty level 3) 
Factorization (Difficulty level 3) 
Common Divisor (Difficulty level 4) 



Fractions 
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(Line-items under the minor-topic of fractions) 
Addition & Subtraction (+/-) with Common Denominator 

(Difficulty level 3) 
+/- with Integers (Difficulty level 4) 
5 +/- without Common Denominator (Difficulty level 5) 

Multiplication and Divisions (*f) with Integers (Difficulty level 5) 
V with fraction (Difficulty level 6) 
Compound Fractions (Difficulty level 6) 
Fraction Reduction (Difficulty level 7) 
1 0 Ratios and Proportions (Difficulty level 7) 

Another example with the minor topic being differential calculus is as follows: 

Calculus (major topic) 
15 Differential calculus (minor topic) 

Fractions (Difficulty level 1) 
Polynomials (Difficulty level 1) 
Exponential Functions (Difficulty level 1) 
Differentiation (Difficulty level 2) 
20 Differentiate a sum (Difficulty level 3) 

Differentiate a product (Difficulty level 3) 
Differentiate a quotient (Difficulty level 4) 
In one embodiment, each line-item has a difficulty level. The bracketed difficulty level 
next to each line-item in the above example indicates how difficult one line-item is 
25 relative to other line-item in the subject, or how significant one is relative to another. A 
line-item whose difficulty level is at a low number implies that it is a relatively easy line- 
item, or that it is a relatively less important line-item. Typically, a student learning a 
subject starts from learning line-items at the lowest difficulty level. 

The lists of items in the above examples are generated based on expert knowledge 
30 on the subject of mathematics. With the proper instruction, such as through reading the 
present specification, generating such lists with the difficulty levels should be obvious to 
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experts in the subject. The more knowledgeable the expert, the more complete the sets of 
items. 

When the student starts learning the subject, all the line-items are classified as un- 
learnt line-items. As the student progresses through the subject, he would have learnt 
5 some of the line-items, and those un-learnt line-items would then be re-classified as learnt 
line-items. 

Different criteria may be used to evaluate if the student has learnt a line-item. 
Generally, learning implies achieving a preset mastery level. This level depends, for 
example, on the subject and the line-item, and is determined by education experts in that 

1 0 subject and in the behavior of human learning. In the present invention, one does not 
have to totally understand every aspect of a line-item before he is considered to have 
learnt that line-item. For example, achieved a passing grade in a line-item may indicate 
having learnt that line-item. In one embodiment, as long as there is an evaluation based 
on a preset level, and the student has reached or exceeded that level, the student is 

15 considered to have learnt that line-item. In another embodiment, as long as there are two 
types of learning materials on a line-item for the student (detailed learning materials and 
summarized learning materials), the present invention assumes that there is a preset level 
in learning that line-item. As the student progresses from the detailed learning materials 
to the summarized learning materials, the student has achieved the preset level in the 

20 detailed learning materials, with the summarized learning materials serving as a review 
for the line-item. 

FIG. 1 shows one embodiment 50 of a line-atem entry E representing a line-item. 
/ For the sake of clarity, that line-item is known as a parent line-item in this example. The 

entry E includes a line-item-root 52, an associated line-item-root list 54, and line-item 
25 attributes 56. 

The line-item root 52 includes the corresponding line-item 64 (the parent line- 
item) and its root, which in one embodiment includes its subject 58, its major topic 60, 
and its minor topic 62. The associated line-item-root list 54 may include one or more 
associated line-item roots, such as 66, with each representing a line-item (a child line- 
30 item) that is related to the parent line-item. For example, if the parent line-item is 

differentiation, the children line-items may include exponential functions, logarithmic 
functions and hyperbolic fiinctions. 
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Each associated line-item-root list 54 denotes the corresponding child line-item, 
its roots, and its association factor, such as 88. The association factor represents the 
degree of relatedness of each child line-item with the parent line-item 64. For example, 
the parent line-item 56 is computer network modeling using Poisson process; the children 
line-items may include the stochastic process and the Guassian process. The stochastic 
process is more related to the Poisson process than the Guassian process. Thus, the 
association factor of the stochastic process has a higher value than the association factor 
of the Guassian process. 

The line-item attributes 56 describe a number of attributes for the parent line-item. 
In one embodiment, the attributes of the parent line-item include, for example, its mode 
70, its difficulty level 72, its mastery level 74, its time-stamp 76, and its review-counter 
78. The mode 70 denotes whether the student has learnt the line-item (the learnt mode), 
or the student has not learnt the line-item (the un-learnt mode). The difficulty level 72 
denotes the level of difficulty of the parent line-item as compared to other line-items. If 
the line-item is a learnt line-item, the mastery level 74 denotes the level achieved by the 
student in learning that line-item; the time-stamp 76 denotes the time when the student 
achieved that mastery level; and the review-counter 78 denotes the number of times that 
line-item has been reviewed. Initially, all line-items are in the un-learnt mode, with the 
mastery levels and the time-stamps not being set, and with the review-counter being zero. 

FIG. 2 shows one embodiment of the computer-aided-educational system 100 of 
the present invention. As an overview of some of its major components, the system 100 
includes a line-item storage-medium 106, which stores the learnt and un-learnt line-items 
of the subject. A selector 102 in the system 100, based on some rules from a rules 
storage-medium 104, selects a line-item from the line-item storage-medium 106. For the 
selected line-item, a learning-material generator generates learning materials for the 
student. In one embodiment, the generator includes a study-plan generator 108, which 
accesses a study-plan from a study-plan storage-medium 1 1 6 for a presenter 1 14 to 
present to a student. In another embodiment, the learning material generator further 
includes a question generator 1 10. The question generator 110 accesses a set of questions 
from a question storage-medium 1 1 8 and sends the questions to the presenter 1 14 to be 
presented to the student. Then, an assessor 1 12 analyzes the student's performance, and 
sends the results to the selector 102, which then selects another line-item for the student. 
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FIG. 3 A shows one embodiment of a system 1 50 implementing the present 
invention, preferably in software and hardware. The system 1 50 includes a server 
computer 152 and a number of client computers, such as 154. Each client computer 
communicates to the server computer 152 through a dedicated communication link, or a 
5 computer network 1 56. 

FIG. 3B shows one embodiment of the client computer 154. It typically includes a 
bus 159 connecting a number of components, such as a processing unit 160, a main 
memory 162, an I/O controller 164, a peripheral controller 166, a graphics adapter 168 
and a network interface adapter 170. The I/O controller 164 is connected to components, 
10 such as a harddisk drive 172 and a floppy disk drive 174. The peripheral controller 166 is 
connected to components, such as a keyboard 176. The graphics adapter 168 is connected 
to a monitor 1 78; and the network interface adapter 1 70 is connected to the network 120. 
The network includes the internet, an intranet, the world wide web and other forms of 
networks. 

15 Different components of the present invention can be in different elements shown 

in FIGS. 3A-B. For example, the presenter 1 14 and the assessor 1 12 can be in a client 
computer; the selector 102, the question generator 110, the study-plan generator 108, and 
the line-item storage-medium 106 can be in the server computer 152. In another 
embodiment, the selector 102, the question generator 1 10 and the study-plan generator 

20 1 08 are also in a client computer. A number of operations in the present invention can be 
implemented by software, which is controlled, for example, by the processing unit 160. 
In yet another embodiment, the number of operations implemented by software can be 
stored in a storage-medium, which can be, for example, the main memory 162 or a CD 
read-only-memory. 

25 FIG. 4 shows a set of steps 200 to implement one embodiment of the present 

invention. First, an un-learnt line-item is selected (step 202), and detailed learning 
materials on the un-learnt line-item is generated (step 204) for the student. Then a learnt 
line-item is selected (step 206), and summarized learning materials on the learnt line-item 
is generated (step 208). The following further describes each of the above steps. 

30 Different methods may be used to select an un-learnt line-item (step 202). In one 

embodiment, the selector 102 first isolates the one or more un-leamt line-items that is at 
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the lowest difficulty level among all the un-learnt line-items. Then out of the one or more 
isolated un-learnt line-items, one is randomly selected. 

In another embodiment, all the line-items of a subject form a difficulty hierarchy, 
with the subject itself at the top of the hierarchy, and the one or more easiest line-items at 
the lowest level of the hierarchy. Selecting an un-learnt line-item starts from the bottom 
of the hierarchy on up. If there are a number of un-learnt line-items at the same level of 
the hierarchy, one of them is randomly selected. FIG. 5 shows an example of such an 
hierarchy with some of the terms in the figure defined as follows: 

Int Integer 

+/- Addition and subtraction 

* Multiplication 

/ Division 

w/ With 

w/o Without 

Fit Fraction 

Comm Denom Common Denominator 

After the un-learnt line-item has been selected, a learning material generator then 
generates detailed learning materials (step 204) for the student. FIG. 6 shows a set of 
steps 250 to generate detailed learning materials in the present invention. The selector 
102 sends to the study-plan generator 108 the selected un-learnt line-item. The generator 
108 accesses (step 252) from the study-plan storage medium 1 16 a set of detailed study- 
plan on the un-learnt line-item. 

In one embodiment, the selector 102 sends the line-item entry E to the study-plan 
generator 108 to retrieve the corresponding study-plan from the study-plan storage- 
medium 1 16. Typically, there are a number of study-plan entries in the storage-medium 
1 1 6, and they can be in the following format: 

(line-item root, mode, study-plan) 
The following serves as examples of a detailed study-plan for differentiating polynomial: 
First, the system teaches the approach to generate derivatives based on the 

basic principle in differentiation, such as: 

df(x)/dx = lim h ^ 0 ((f(x+h) - f(*))/h) 
11 
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Then the system teaches the generalized equation, such as: 
((dIaixi)/dx) = (Si*aixi-l) 

Finally, the system teaches the importance of and the way to find optima 
and minima by solving the following equation: 
5 ((dSajxi)/dx) = 0 

Based on the line-item root and its mode in the line-item entry E, and with one set of 
detailed study-plan per un-learnt line-item, the generator 108 retrieves from the study- 
plan storage-medium 1 16, the corresponding detailed study-plan. Creating detailed study- 
plans for a subject should be obvious to experts in the subject, and will not be further 
10 discussed in this application. 

The generator 108 then sends the retrieved study-plan with the line-item entry E to 
the presenter 114, which presents (step 254) the study-plan to the student. The study-plan 
can be a document containing studying materials for the un-learnt element. The 
document can be arranged as a list of screens, each of which covering part of the material. 
15 The presenter 1 14 typically includes the monitor 178, which presents the study plan to the 
student, who can go from one screen to another with the keyboard 1 76, or a mouse 
device. In another embodiment, the study plans are broadcast through a radio, or 
presented through a facsimile machine. 

In this embodiment, as long as the study-plan has been presented to the student, it 
20 is assumed that the student has mastered the line-item. After the presentation, the study 
plan generator sends a mission-accomplished signal with the line-item entry E to the 
assessor 1 12. Presumably, the student, by going through the study plan, should have 
learnt the line-item. 

Then, the assessor 1 12 verifies the mission-accomplished signal, and sends it with 
25 the line-item entry E to the selector. The selector, based on rules in the rules-storage 
medium 104, determines that the un-learnt line-item has been learnt. In the present 
situation, the rule is simply that once the study-plan has been presented, the line-item has 
been learnt. The selector 102 then re-classifies the mode in the entry E from un-learnt to 
learnt, stores the time when the line-item is re-classified in the time-stamp, and stores an 
30 identifier, such as "pass", in the mastery level. The line-item with its modified entry E is 
stored in the line-item storage-medium 106. 



12 



WO 99/24953 PCT/US98/23979 

In another embodiment, after presenting to the student the detailed study-plan, 
questions are generated (step 256) and presented (step 258) to the students. Typical, 
students gain a better understanding on a subject through actively working on questions, 
than just through passively reading study plans. 

In this embodiment, after the student indicates that he has finished studying the 
study-plan, the generator 1 10 accesses a set of questions from the question storage 
medium 1 18, and sends the questions to the presenter 1 14 to be presented (step 258) to 
the student. In one embodiment, each question is embedded in a question entry, which is 
of the following format: 

(line-item root, mode, question-body, answer). 
The term "question-body" describes the body of a question. The following serves as an 
example: 

Subject: Mathematics. 

Major-topic: High School Algebra. 

Minor-topic: Fraction. 

Line-item: +/- with common denominator 

Mode: Un-learnt 

Answer Question-body 
28/37 What is the sum of 2/37, 3/37, 8/37 and 15/37? 

-2/43 17/43 -25/43 + 6/43 = ? 

The selector 102 sends to the question generator 1 10 the line-item entry E. The 
question generator 1 10, by matching the line-item root and the mode in the line-item entry 
E with the line-item root and the mode in the question entries, selects one or more 
question entries. For each question-entry selected, it is sent to the assessor 1 12, with its 
question-body sent to the presenter 1 14 to be presented to the student. The student 
answers the questions, for example, by entering his answers through the keyboard 1 76 to 
the presenter 1 14, and the results are sent to the assessor 1 12. In another embodiment, the 
correct answers to the questions together with the student's answers are sent by the 
presenter 1 14 to the assessor 112. 
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In yet another embodiment, the learning materials only include questions. For 
example, the learning materials are for the un-learnt line-item of differentiating 
exponential functions. A number of questions are generated, including the question on 
expanding an exponential function based on Taylor expansion, the question on 

5 differentiating the Taylor-expanded exponential function, whose answer is the original 
Taylor-expanded exponential function, and the question on differentiating the exponential 
function, whose answer is the exponential function. The presenter 1 14 bypasses study 
plans, and receives the questions, which are presented to the students. 

Note that the formats of the learning materials may change as the student is 

10 learning. The student can learn one line-item based on questions, and another based on 
study-plans. As an example, in the above example of a minor topic being differential 
calculus, for the different line-items, all of them can be learnt through both study-plans 
and questions, except for the line-item of differentiation, which is typically learnt through 
a study-plan. That study-plan covers the general differentiation concept, such as the 

15 following: 

df(x)/dx = lim h _> 0 ((f(x+h) - f(x))/h) 
For the embodiment where the learning materials include questions, the assessor 
1 12 assesses (step 260) the student's answers by comparing the students answers to the 
answers in the question entries previously received. Through the analysis, the assessor 
20 112 determines the student's learning progress. 

In one embodiment, the assessor 1 1 2 receives question entries from the question 
generator 1 1 0. The assessor 1 12 also receives the student's answers with their 
corresponding question-bodies from the presenter 1 14. For each question, using the 
question-body as a key, the assessor 1 12 evaluates the student's answer by matching it 
25 with the answer in the corresponding question-entry. 

Note that generating the questions should be obvious to experts in the subject, and 
will not be further discussed in this application. 

After the assessment, the assessor 1 12 sends the student's assessed results on the 
line-item back to the selector. If the study materials include questions, the student's 
30 assessed results can be in the form of a score, or a grade. If the study materials are study- 
plans only, the student's performance is an indicator depending on whether the plans have 
been presented to the student. The selector 1 02 based on rules stored in the rule storage- 
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medium 104 determines if the assessed results are satisfactory or not (step 262). For 
example, a score of 70 or higher is satisfactory, and 69 or lower is unsatisfactory; a grade 
of C or better is satisfactory, and D or lower is unsatisfactory; or a study-plan, after 
presentation, is considered as satisfactory. If the assessed results are unsatisfactory, then 

5 the system proceeds as if the student has not learnt that line-item. 

If the selector 102 is satisfied with the student's assessed results in the selected un- 
learnt element, as discussed above, the selected un-learnt line-item is re-classified as a 
learned line-item, by having its mode field changed to the learnt mode, with the time 
when this occurs stored in the time-stamp, and with the student's assessed results stored in 

10 the mastery level, such as 74. The modified entry E is re-stored in the line-item storage 
medium 106. 

In one embodiment, the assessor 1 12 passes an assessment entry back to the 
selector 102. The format of an assessment entry is similar to that of a line-item entry, 
except the mastery level field is replaced by the student's assessed results in the 

1 5 corresponding line-item. 

After the generation of the detailed learning materials (step 204), the selector 102 
selects a learnt line-item (step 206) based on one or more learnt-line-item-selection rules 
stored in the rules storage-medium. FIG. 7 shows examples of factors affecting such 
rules to select a learnt line-item in the present invention. 

20 In one embodiment, the learnt line-item is selected depending on the time elapsed 

from the time when the line-item is re-classified from the un-learnt to the learnt mode 
(box 300). The selection process dis-favors the line-item that has just been re-classified 
because presumably the line-item is still fresh in the student's mind. As time passes, the 
student's memory begins to fade, and should be refreshed. Typically, a student starts to 

25 forget what he has learnt a number of weeks, such as three, after he has learnt it. Thus, in 
one embodiment, the selection process starts favoring reviewing the line-item that has 
been re-classified three weeks ago. 

In another embodiment, the selection process dis-favors the line-item that has 
been re-classified long time ago because presumably the student has learnt other more 

30 difficult line-item during the interim period. This is similar to a student learning a subject 
in a semester. After the end of the semester, the student stops reviewing the subject, and 
starts studying a more difficult subject. Typically, a semester is about three months or 
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twelve weeks. Thus, in one embodiment, the selection process begins to dis-favor 
reviewing the line-item that has been re-classified more than twelve weeks ago. 

In one embodiment, the above elapsed time dependency in selection is embedded 
in a rule that is represented by the following equation: 

Weight w(t) = k * e-at * (l-e-*)b 
where k, a, and b > 0. Here the time t starts counting from the time when the line-item is 
re-classified. Initially, the weight function w(t) increases as time t increases, peaking with 
time being equal to In ((a+b)/a). From that point onwards, the function decreases as time 
further increases. The values for a and b are chosen to determine the position of the peak, 
and the value for k is used to adjust the variance of the function. 

The bigger the weight of a line-item, the better the chance for that line-item to be 
selected. As an example, the weight is chosen to peak when t is between three weeks (or 
21 days) and twelve weeks (or 84 days), which averages to 53 days. Thus, w(t) peaks 
around 53 days, implying that: 

In ((a+b)/a) = 53. 

The above exponential weight function is just an example representing the 
dependency on time. Other weight functions are also applicable, such as the Chi's Square 
functions. 

In another embodiment, selecting a learnt line-item depends on the mastery level 
achieved by the student (box 302) when the line-item was changed from the learnt mode 
to the un-learnt mode. The lower the grade or the level achieved by the student, the 
higher the chance to select that line-item because the educational system should reinforce 
the student's weaker areas. Again, this can be represented by a weight function, which 
depends on the mastery level achieved by the student. 

In yet another embodiment, selecting a learnt line-item depends on the 
characteristics of the line-item, such as the difficulty level of the line-item (box 304). The 
more difficult the line-item, the higher the probability to select that line-item because it is 
more likely for the student to be weak in that line-item, or that line-item is more 
significant than other line-items. Again, the dependency on difficulty level can be 
represented by a weight function, which tracks the difficulty level of the line-item. 

For another different embodiment, selecting a learnt line-item depends on whether 
that line-item is related to the most recently selected un-learnt line-item (box 306). This 

16 
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embodiment favors a learnt line-item that is related to the un-learnt line-item because the 
student may form the relationship himself through working on both of them. For 
example, the un-learnt line-item just learnt is differentiating exponential functions. Then 
one learnt item to review may be the hyperbolic sine and cosine functions. The idea is for 
the student to form the relationship himself that differentiating hyperbolic sine is the 
hyperbolic cosine, and vice versa. Typically, a student enjoys learning a subject and 
remembers it better if he thinks that he, by himself, has developed some insights in the 
subject. 

One way to identify related line-items is from the line-item entry shown in FIG. 1 . 
Each line-item in an associated line-item root, such as 66, is related to the line-item in the 
parent line-item root 52. The degree of relatedness is represented by the association 
factor, such as 88. The line-item that is more closely related to the parent line-item has a 
larger association factor than a line-item that is not as closely related. Line-items not 
listed in the associated line-item root-list are either not related or too remotely related to 
the line-item in the parent line-item 56. In one embodiment, the weight w depends on the 
association factor. 

In yet another embodiment, selecting a learnt line-item depends on whether that 
learnt line-item has been selected before, as shown in the review-counter (box 308). The 
value in the review-counter does not include one or more prior selections of the line-items 
when the line-item was in its un-learnt mode. The selection process discourages re- 
selecting a line-item that has been selected before. One way to achieve this result is to 
multiply the weight factor affecting which line-item to select by 

l/(the value in the review-counter of that line-item + 1). 
The higher the factor, the lower the probability that the corresponding line-item is 
selected. Thus, the above equation reduces the probability of selecting a line-item that 
has been reviewed before. 

One or more of the above factors may be combined in a learnt-line-item selection 
rule to influence the selection of a learnt line-item. The combination process can simply 
be based on a weight factor that depends on one or more of the above factors. 

After selecting the leamt line-item, summarized learning materials are generated 
(step 208) to be presented to the student. Generally, summarized learning materials are 
more concise than the detailed learning material. In one embodiment, summarized 

17 
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learning materials do not include any questions to test the student. In terms of the 
process, in one embodiment, the selector 102 sends the line-item entry E to the study-plan 
generator 108 to retrieve the corresponding summarized study plan from the study-plan 
storage-medium 116. Typically, there are a number of study-plan entries in the storage- 

5 medium 1 16, and they can be in the following format: 
(line-item root, mode, study-plan) 
If the mode is a learnt mode, the study-plan is a summarized study-plan. The following 
serves as an example of a summarized study-plan for differentiating polynomial: 
The system teaches the generalized equation: 

1 0 ((dZajx^/dx) = (Zi * ajx*- 1 ) 

The system also teaches the following equation, which is used to find 
optima and minima: 

((dZajxtydx) = 0 

Based on the line-item root and its mode in the line-item entry E, and with one 

1 5 summarized study plan per learnt line-item, the generator 108 retrieves from the study- 
plan storage-medium 1 16, the corresponding summarized study-plan. Creating 
summarized study-plans for a subject should be obvious to experts in the subject, and will 
not be further discussed in this application. 

In another embodiment, summarized learning materials also include a number of 

20 questions to test the student. These questions may be different from those in one 
embodiment of the detailed learning materials. The answers are used to assess the 
understanding level of the student in that line-item. 

In the above embodiments, the detailed learning materials are generated before the 
summarized learning materials. In another embodiment, detailed learning materials for 

25 an un-learnt line-item are generated (step 204) after the generation of the summarized 
learning materials (step 208). This allows the student to review a learnt line-item before 
studying detailed learning materials on an un-learnt line-item. This different approach is 
applicable if the learnt line-item is related to the un-learnt line-item. A review helps to 
prepare the student for the un-learnt line-item. 

30 In one embodiment, after a line-item has been reviewed, its review factor 78 is 

incremented by one. For that embodiment, the selection of a line-item to be reviewed is 
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inversely proportional to the review factor. Thus, the probability of again selecting a line- 
item which has previously been reviewed is reduced. 

The steps shown in FIG. 4 can be repeated. FIG. 8 shows a set of steps to 
implement another embodiment of the present invention, where there is repetition in 
selecting line-items. This embodiment starts from selecting an un-learnt line-item (step 
352). Then, a line-item probability value is accessed (step 354) from the rules storage 
medium 104 to select a line-item (step 356). That value can be set by an instructor 
through a councilor 120. Note that there may be a number of steps in between the step of 
selecting (step 352) an un-learnt line-item and the step of accessing a line-item probability 
value (step 354). Anyway, typically, when a student is learning a subject, he starts with 
working on an un-learnt line-item. But it is not necessary to start the process with 
selecting an un-learnt line-item. 

FIG. 9 shows one embodiment of a set 400 of steps to select an un-learnt line-item 
or a learnt line-item in the present invention based on the line-item probability P. From 
the value P, an un-learnt line-item is selected (100*P)% of the time and a learnt line-item 
(100*(1-P))% of the time. Then, for the selection process, the selector 102 generates 
(step 402) a random number between 0 and 1 , and compares (step 404) the random 
number with P. If the number is smaller than P, the selector selects (step 406) an un- 
learnt line-item. If there is no more un-learnt line-item, the student has mastered the 
subject, and can proceed to another, or the system can select learnt-line-items for review. 
If the number is bigger than or equal to P, the selector selects (step 408) a learnt line-item. 
If there is no learnt line-item, the selector 102 selects (step 412) an un-learnt line-item. 

In one embodiment, one characteristic of the probability P is that the higher the 
value of P, the more likely the selector 102 selects an un-learnt line-item. Such a 
system/method favors teaching new concepts. The smaller the value of P, the more likely 
the selector 102 selects a learnt line-item. Such a system/method favors reviews to 
reinforce the understanding of the learnt line-items before attempting new concepts. 

In another embodiment, the selector 102 automatically selects the probability P, 
which changes as the student is working through the subject. The probability P depends 
on its previously selected item. To illustrate such an embodiment, suppose P(n) is the 
line-item probability after selecting n line-items, then 
P(n+l) = F(P(n)) n>0 
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where F is a function operating on P(n). The initial value P(0) is randomly set by the 
system to be between 0 and 1 . The idea is that the invented system tends to switch to 
teach a learnt line-item after teaching one or more un-learnt line-items, and vice versa. 
To increase the probability of choosing a learnt line-item after selecting one or more un- 
learnt line-items, the function F produces a value smaller than its argument P(n). 
Similarly, to increase the probability of choosing an un-learnt line-item after selecting one 
or more learnt line-items, the function F produces a value larger than its argument P(n). 
The following serves as an example for this embodiment, with PI being the previously 
selected line-item: 

P(n+1) = Max(0, P(n)-c) if PI is an un-leamt line-item. 
P(n+1) = Min(l, P(n)+d) if PI is a learnt line-item. 
"Min" denotes "The minimum of." "Max" means "The maximum of." 
Both c and d are constants between 0 and 1 ; they are chosen to reflect how 
quickly the invented system selects a different type of line-item. Each 
time a learnt line-item is selected, P is increased by an amount of d, until P 
reaches 1 . Similarly, each time a un-learnt line-item is selected, PI is 
reduced by an amount of c, until P becomes 0. 
After the line-item is selected, if it is an un-learnt line-item, detailed learning 
material is generated (step 358). If the student is satisfactory in learning the detailed 
learning material, the un-learnt line-item is re-classified (step 362) as a learnt line-item. 
On the other hand, if the selected line-item is a learnt line-item, then summarized learning 
material is generated (step 360). The process then repeats to select another line-item. 
The generation of learning materials and the re-classification process are as described 
above. 
Conclusion 

In the above examples, initially the modes of all line-items are classified as un- 
learnt. In one embodiment, even initially, the modes of some of the line-items are 
classified as learnt, which can be done by the instructor through the councilor 120. In 
another embodiment, initially all line-items are classified as learnt line-items. In such an 
embodiment, the student has previously learnt the subject, and the embodiment is just for 
reviews, which is suitable for one to prepare for an examination. Thus, only summarized 
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learning materials are generated for the student, with no initial step of selecting an un- 
learnt line-item. 

In the specification, learning materials can be study-plans, or questions, or study- 
plans with questions. In the embodiment where the learning materials only include study- 

5 plans, at least one line-item's learning materials include questions to more accurately 
assess the student's level in learning that line-item. 

In a number of the above embodiments, rules are accessed from the rule storage 
medium. In another embodiment, those rules have previously been embedded in the 
programs implementing the present invention. With rules already embedded in the 

10 program, there is no need for accessing the rules. 

Finally, a number of the above embodiments involve a number of steps that seem 
to have a specific sequence. For example, the step of generating detailed learning- 
material for an un-learnt line-item can comprise the steps of generating a detailed-study 
plan for the un-learnt line-item, presenting the detailed-study-plan to the student, 

15 generating questions on the un-learnt line-item for the student, and presenting the 
questions for the student. In the present invention, those embodiments include 
reshuffling the steps as long as there is no inherent dependency among those reshuffled 
steps. For example, one can reshuffle some of the steps of generating detailed learning- 
material for an un-learnt line-item to read: generating a detailed-study plan for the un- 

20 learnt line-item, generating questions on the un-learnt line-item for the student, presenting 
the detailed-study-plan to the student, and presenting the questions for the student. 

Other embodiments of the invention will be apparent to those skilled in the art 
from a consideration of this specification or practice of the invention disclosed herein. It 
is intended that the specification and examples be considered as exemplary only, with the 

25 true scope and spirit of the invention being indicated by the following claims. 
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CLAIMS 



We claim: 



1 1 . A computer-aided learning method to reinforce a student's learning of a subject 

2 through reviews, the subject including a number of line-items, with at least one being a 

3 learnt line-item, which is a line-item that the student has achieved a preset level of 

4 learning, and with the line-items being stored in a line-item storage-medium, the method 

5 comprising the steps of: 

6 selecting a learnt line-item from the line-item storage-medium based on one or 

7 more line-item-selection rules; and 

8 generating summarized-learning materials on the learnt line-item for the student; 

9 such that although the student has achieved a preset level in learning a line-item, 

10 that learnt line-item still might be selected with its learning materials generated for the 

1 1 student so as to reinforce the student's learning of the subject through reviews. 

1 2. A computer-aided learning method to reinforce a student's learning of a subject 

2 through reviews, the subject being divided into un-learnt and learnt line-items, with a 

3 learnt line-item being an item the student has achieved a preset level in learning, and with 

4 the line-items being stored in a line-item storage-medium, the method comprising the 

5 steps of: 

6 selecting a line-item from the line-item storage-medium based on a probability 

7 value; 

8 if a learnt line-item is selected, then generating summarized-learning materials of 

9 that learnt line-item for the student; 

10 such that although the student has achieved a preset level in learning a line-item, 

1 1 learning materials of that learnt line-item might still be generated for the student so as to 

12 reinforce the student's learning of the subject through reviews. 

13. A computer-aided learning method as recited in claim 1 wherein: 

2 the subject includes at least one un-learnt line-item, which is a line-item that the 

3 student has not achieved the preset level in learning that line-item; and 
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4 the method further comprises the step of selecting an un-learnt line-item from the 

5 line-item storage-medium before the step of selecting a learnt line-item. 

1 4. A computer-aided learning method as recited in claims 1 , 2 or 3 wherein the step 

2 of selecting a learnt line-item depends on at least one of the following factors: (the time 

3 elapsed from the time when the student has achieved the preset level for that line-item, 

4 the difficulty level of the learnt line-item, and the level achieved by the student in 

5 learning that learnt line-item). 

1 5. A computer-aided learning method as recited in claims 2 or 3 wherein the step of 

2 selecting a learnt line-item depends on whether the learnt line-item is related to the 

3 selected un-learnt line-item so as to reinforce the student's learning of the subject through 

4 association. 

1 6. A computer-aided learning method as recited in claims 1 , 2 or 3 wherein the step 

2 of selecting a learnt line-item depends on whether that learnt line-item has been selected 

3 before so as to discourage selecting a learnt line-item that has been selected before. 

1 7. A computer-aided learning method as recited in Claims 1 , 2 or 3 further 

2 comprising the steps of: 

3 generating questions on a line-item; 

4 presenting the questions for the student; and 

5 assessing the student's answers to those questions to determine if the student has 

6 achieved a preset level of learning that line-item. 

18. A computer-aided learning method as recited in Claims 2 or 3 further comprising 

2 the steps of: 

3 if an un-learnt line-item is selected, then 

4 generating detailed learning-material for the un-learnt line-item, and 

5 if the student has achieved a preset level in learning that line-item, then re- 

6 classifying the un-learnt line-item as a learnt line-item; and 

7 repeating from the step of selecting a line-item. 
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1 9. A computer-aided learning method as recited in Claims 1, 2 or 3 further 

2 comprising the step of transmitting the generated summarized-learning materials through 

3 a network to be received by a computer. 

1 10. A computer-aided learning apparatus to reinforce a students learning of a subject 

2 through reviews, the subject including a plurality of line-items, with at least one being a 

3 learnt line-item, which is a line-item that the student has achieved a preset level of 

4 learning, the apparatus comprising: 

5 a selector configured to select a line-item from the plurality of line-items based on 

6 at least one line-item-selection rule; and 

7 a learning-material generator coupled to the selector configured to generate 

8 learning materials of the selected line-item for the student; 

9 such that although the student has achieved a preset level in learning a line-item, 

10 that line-item might still be selected with its learning materials generated for the student 

11 so as to reinforce the student's learning of the subject through reviews. 

t 11. A computer-aided learning apparatus to reinforce a student's learning of a subject 

2 through reviews, the subject being divided into un-learnt and learnt line-items, with a 

3 learnt line-item being an item the student has achieved a preset level in learning, the 

4 apparatus comprising: 

5 a selector configured to select a line-item based on a probability value; 

6 a learning-material generator configured to generate summarized-learning 

7 materials for a learnt line-item selected for the student; 

8 such that although the student has achieved a preset level in learning a line-item, 

9 learning materials of that learnt line-item might still be generated for the student so as to 

10 reinforce the student's learning of the subject through reviews. 

1 12. A computer-aided learning apparatus as recited in claim 10 wherein: 

2 the subject includes at least one un-learnt line-item, which is a line-item that the 

3 student has not achieved the preset level in learning that line-item; 
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4 if the line-item selected is a learnt line-item, then the generator generates 

5 summarized learning-materials for that learnt line-item; and 

6 if the line-item selected is an un-learnt line-item, then the generator generates 

7 detailed learning-materials for that un-learnt line-item. 

1 13. A computer-aided learning apparatus as recited in claims 10, 1 1 or 12 further 

2 comprising: 

3 a storage-medium for storing the line-items; 

4 a study-plan generator for generating learning materials; 

5 a presenter for presenting to the student; 

6 a question generator for generating questions; and 

7 an assessor for assessing the student's answers to questions. 

1 14. A computer-aided learning apparatus as recited in claims 10, 1 1 or 12 wherein the 

2 selector selects a learnt line-item depending on at least one of the following factors: (the 

3 time elapsed from the time when the student has achieved the preset level for that line- 

4 item, the difficulty level of the learnt line-item, and the level achieved by the student in 

5 learning that learnt line-item). 

1 15. A computer-aided learning apparatus as recited in claims 1 1 or 1 2 wherein the 

2 selector selects a learnt line-item depending on whether the learnt line-item is related to 

3 the selected un-learnt line-item so as to reinforce the student's learning of the subject 

4 through association. 

1 16. A computer-aided learning apparatus as recited in Claims 1 0, 1 1 or 1 2 wherein the 

2 generated summari zed-learning materials are transmitted through a network to be 

3 received by a computer. 
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